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The Pearson and likelihood ratio statistics are ' 
frequently used for assessing the absolute fit of probability models. 
Researchers are often interested in comparing fits provided by 
different models which may have a subsuming or non-subsuming' 
relation. A subsuming relation exists when the parameters of the 
reduced model form a Subset of those contained in the subsuming, or 
full, model. A non-subsuming relation exists when the defining 
parameters in neither model form a subset of those defining the other 
model. A general "mixture" probability model which incorporates two 
non-subsuming probability models and a strategy for assessing fit 
provided by each of the component models is described. A detailed 
description of the strategy for selecting a preferred model is 
outlined. Applications of the suggested procedure are considered for * 
the special case in which the Rasch latent trait model and Latent 
State Mastery model are compared as to relative preference when data 
are generated from each .model. (DWH) 
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ABSTRACT , 

V * 

A "mixture" probability model; which incorporates two psycho- 

, ''metric models from separate parametric families is introduced and the 

application of this model in selecting a preferred component model 

is described. Example applications of the suggested procedure are 

i 

considered for the special case in which the Rasch latent trait model 

» 

and the Latent State Mastery model are compared as to their relative 
preference when data are generated from each of these models. 

<: 

* * \ 



Introduction 

s 

'Within the j£alm of probabilistic modeling, researchers are 
frequently interested in ascertaining how well investigated models 
account for manifest data. Two frequently used statistics for - - 
assessing the absolute' fit of probability models are the 
Pearson and the 1 likelihood ""ratio statistics. These statistics are 
defjned respectively in equations (1) and (2). 
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(2). 

where 

th 

' * 0.j =' observed frequency for the i response category 9 

I 32 expected frequency for the 1 th response category, and 
n f 32 number of response categories. 
Both of these statistics are asymptotically distributed as chi- 
square (given np>k+l) with; degrees of freedom \i 

df f = n r - k - 1 (3) 

where k is the number of independent parameters defining the model in 
question. 

Note that the observed and expected frequencies on which these 
statistics are based may correspond to either "total scores" or item 
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"response "patterns. " The preferability of thesevtwo- classes of 

I . 1 
response categories will depend on such factors as number* of . re- 

spondents, N, number of items, m, and number of parameters, k, to 

be estimated^ 

In addition to assessing -the absolute fit of models to d^tjfa, re- 
searchers are frequently interested in comparing the fits provided by 
different models. When comparisons between pairs of models are of 
interest, two situations may arise: (1) the models have a subsuming 
relation, or .(2) the models have a non-subsuming relation* 

A subsuming relation exists between tyo models when the parameters" 
of one model* (the reduced model) form a subset of those contained in the 
'second more complex model (the subsuming 6r full model). When models 

c 

have a subsuming delation, it is possible to assess relative fit of 
the reduced model to thej ful 1 model (i.e., to determine whether -the 
reduced model fits as well as the full model). This may be accomplished 

5" * ■ 

/» 1 

by using the difference in*the likelihood ratio statistics defined in 

T 

(2) for the full (I) and the reduced (II) models, 

v | 

2 ^ 2 V 2 

X I) ' % X LR(II) ~ X LR(I) (4) 



The difference statistic, \ $ .is asymptotically distributed 
as chi-square (given that the full model holds) with degrees of freedom 

tffp-kj.-kj, * (5) 

where kj and kjj are respectively the number of independent parameters 
in models I and IT. 
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Two models have' a non-subsuming relation when the defining 
parameters, in neither model form a ^subset of those defining the other 
model. A problem arises if a researcher is interested in comparing 
two models that have a( non-subsuming relation, since the difference 
statistic defined in (4) is not distributed as chi-square. Thus, al- 
^e"rnli^e~priocedures are" : needed fbr~assessing the relative adequacy of 
two models with respect to fit. > 

The probfem of comparing non-subsuming models falls^ within ttie t 
context of discriminating between models from separate families. A ^ 
theory of hypothesis testing for separate families has been developed 
by Qox (1961, 1962) and extended by Atkinson (1970):, whose general 
approach we follow! 

The purpose of this paper is to introduce a general "mixture" . 
probability model which incorporates two non-subsuming probability . 
models and to present a strategy for assessing fit provided by each of 
the component models. 

The Mixture Model 

pe general model, M^, presented in this paper incorporates two 

al ternati ve non-s ubsumi ng models (I and II) as weighted components of 

the full model. Under the Mixture model, the probability of the occur 

of the 4^ h response category is defined as 

/ • l->.' X. 
9, 0 

♦i - * 1 (6) 



t 

-4- ■ « 
In this equation', G.and ^ respectively represent probabilities for 
the i response category under the non-subsuming models I and II. 

• ' . ' i. 

X represents a "mixture" .parameter "which designates the relative* 
contribution which each component model makes to the probability, Vi . 
Notice that when X is set at zero or one, the full model reduces to 
model I .or model II respectively. 

Estimation and Fit of the Mixture Model 

■ To' obtain maximum likelihood estimates of the parameters which 
define the component latent structure models, iterative methods such 
as the Newton-Raphson procedure (see Bock and Lieberman, 1970) or the 
iterative proportional fitting method(see Clogg, 1977) are frequently 
employed. Similarly, a conditional maximum likelihood estimate of \ 

may also be obtained through the, use of iterative procedures (see Conte 

1 . 

and deBoor, 1972). 

V 

A test for the absolute fit of the Mixture model, \> can be per- 
formed by means of a likelihood ratio statistic, as defined in 

(2), where E 1 ■ NM^.* This statistic is distributed asymptotically as 
chi-square with degrees of freedom equal to n r - kj - k n - 2 (given 
that estimates of the parameters defining Y 1 ar e independent). 

Strategy for Assessing" Fit 

In this section a stagewise strategy for selecting a preferred 
model is presented. Under this strategy it is possible to reach any 

c 

one of the following conclusions related to model preference based on 
fit: 



(a) model I* provides acceptable fit, and is preferred, 

(b) model II provides acceptable fit, and is preferred, 

' (c) both models provide acceptable fit, but neither is preferred, or 
jfd) both models provide unacceptable fit. 

A detailed description of the strategy for selecting a preferred 
model follows, with the flow diagram in Figure 1 reflecting this strategy 

I. , -Assess the absolute fit of the Mixture model, M x , by means of the • \ 

statistic in equation (2). 

A. If absolute fit is obtained,- go to II. 

B. If absolute, fit is not obtained, STOP. Conclude that neither 
Model I nor Model II (separately or combined) are acceptable 
for describing the data. 

II. (From I.A) Assess the relative fit of Model I and Model II to the 
'Mixture model by means of the difference chi-square, %$>, as defined 

in. (4). ■ n 

A. If both Model I and Model II provide acceptable fi-t, go to HI. 

B. If only Model I provides acceptable relative fit, go to IV. 

i 

C. If only Model I provides acceptable' relative fit, go to V. 

D If ineither Model I°nor Model II provide acceptable relative fit, 
STOP. Conclude" that neither Model I -nor Model II are adequate 
to account for the data. 

III. (From II. A) Assess Model I and Model* II with respect to their 
absolute fits with equation (-2)-. -, - ■ 

A. If only Model II provides acceptable fit, STOP. Select 
Model II. 
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Figure 1. 'Flow diagram of the proposed strategy for selecting 
a 'preferred model . 



"B. Jf only" Model I provides acceptable fit, STOP. .Select Model I; 
, C. If. both Model I and Model II provide acceptable fit, STOP.- " - 
Conclude that both Model I /and Model II are acceptable models 
but neither is "preferred" over the other model. 
D-. If neither model provides acceptable, absolute -fit, ( STOP. 
Conclude that" neither model is acceptable. 
IV. (From II. B) Assess the absolute fit of Model II with equation (2). 

A. If absolute fit is obtained, STOP. Select Model II. 

B. If absolute fit. is not obtained, STQP. Conclude that neither*- 
model is acceptable. . .' 

V. (From II. C) /Assess the absolute fit of Model I with equation (2)., 

A. If absolute fit is obtained, STOP. Select Model I. 

B. If absolute fit is not obtained, STOP. Conclude that neither 
' model is acceptable. 

/ Example Applications 

In the area of mastery assessment two classes of latent structure 
models have been proposed. These have been called Continuum models and 
State models (see Meskauskas, 1976). For Continuum models,, trait ' 

CP 

acquisition is assumed to be gradual and ma'stery is viewed as an interval 
on a continuum while for State models, trait acquisition is conceived asy 
an "all-or-none" process and 'mastery is viewed as the presence of trait 
acquisition.. Two models which fall respectively within the Continuum 
and State model classes (and which dp not have a subsuming relation) are 
the^Rasch model (Wright and Stone, 1979) and thi Latent State Mastery 

11 . 



(LSM) model (Macready and Dayton,- 1977). This section of the paper 

is. devoted to the application of the proposed strategy with simulated 

i • » 

• „data, for selection between %he Rasch and LSM models. 

For both the Rasch and LSM, models, the probability of the 

th 

occurrence. o*f the i response pattern may be designated as the product, 
of item response probabilities (assuming .local- indepencence). 

th 

where ji- " the' f response pattern , 

I m « the number of dichotomously scored) items, and 

(l when a correct response is encountered 
h - \ for the g th itemi within the i tn response pattern 

' (p otherwise , 

* For the .Rasch model > the prdbabi.l i ty of a correct response to item 

g over all persons is 



■X 



N 



exp [i < S - b )] 



where 



p - i "^T L j • b 
8 I Z : ~ (8)' 

I 

x< «,the latent ability of the y examinee, 

b„ « difficulty of the g item,, and 
9 • y 

a « common -discrimination factor for all items* 

» 

Correspondingly, for the LSM model, the probability, P , is 

* • 

defined as • * 

P g-(V U)+{P 9.^' 0 "^ ); * (9) - 
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— sa> -9- 
where 

p'' u = the conditional probability that a master (M) answers 
g.M > 

ttem_g-cor-reetlyi 



p = the conditional probability that a non-master (H) • 
g.M 6, 

- answers item g correctly, and 

* A = the latent proportion of, masters (thus, 1 is 

the latent proportion of ^on-masters). 

In equation (6), P(u..) is represented as -fcj-fon the Rasch model and 



0. for the LSM model. 



Data Generation 



Twfllsets of simulated data were considered in this paper. The • 
first set of data was based on the Rasch model for a sample of N = 100 
simulated subjects responding' to m = 5 items. The itej^ difficulties 
-used in'"generatT on were uniformly distributed (-.5, -.45, 0.0, .45, .9) 
and the discrimination factor (a") was set^equal to .85. The latent 
trait* x., was randomly generated from a normal population with a mean 
of 0.0 and standard deviation of 1.0. „, 

The second set of simulated data was based on' the LSM model , here 
also with a sample of N=100 simulated subjects responding to m = 5 items. 
Under this model, the five conditional probabilities for positive item 
responses from non-masters, P g ^ were set at .99, .98, .97, .96, and .95. 

For each set of simulated data, maximum likelihood estimates of the 
parameters defining each model were independently estimated through the 
use of separate programs employing Newton-Raphson algorithms (see Wright, 
Mead, and Bell, 1979; and Dayton and Macready, 1976). Conditional upon 



these estimates, a maximum likelihood estimate of K was obtained using 
the method of bisection (Conte and deBoor, 1972). In addition, models 
were ju3ge"d as^ ^ovfding^atceptable absolute (or relative*) fit 1-f — j 

The results related to the data generated from the Rasch model are 
presented cin Table 1. 

Jable 1 



Results for tests of fit based on data 
generated from the Rasch model 

* 

2 

Assessed Condition < ^ df p- value 



Absolute fit of. M . 


22.42 v ' 


14 


.930 


Relative fit of Rasch 


7.17 


12 


.154 


Relative fit of LSM 


9.34 


6 


.845 


Absolute fit of Rasch 

<• 


29.59 


26 


.715 


Absolute fit of LSM 


31.76 


20 ' 


.954 



Following the detailed strategy for selecting a preferred model, the 

absolute fit of the Mixture model, M , was first assessed (at Stage I). 

K * x 

Since the observed /Lu for the Mixture model suggested adequate fit, 
the' relative fits for both "the Rasch and LSM models to ths mixture model 
were considered (at Stage II). Because each of these models provided 
acceptable relative fjt to the Mixture model, the^j were their assessed vith 
respect to their absolute fit (at Stage III). Only the ^u*for the 
Rasch model failed to Exceed the ,05 critical value. Hence, the Rasch 
model "was selected as the preferred model. c 

Corresponditig results for the data set that was generated -from the 
LSM model are presented in Table 2. 

U4 
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Table'2 



Results T f or— tests. -ofJHJU>AS.ei_pn_. data 

generated from the LSM 'model? 



Assessed Condition 




df 


p- value 


Absolute fit of M x 


* 17.19 

4 


14 


. .754 


Relative fit of Rasch 


22.03 


12 


.963 


Relative fit of LSM 


1.41. 


6 


!035 


Absolute fit of Rasch 


39.22 


26 


.954 


Absolute fit of LSM 


18.61 


20 


.453 ■ 



As in the previous example, the Mixture model provided acceptable 
absolute fit (at Stage I). Thus, (at Stage* II) both the Rasch and LSM 
models were assessed with respect to their relative fits to the Mixture' 
model. Here, only the LSM model provided acceptable relative fit. For 
this reason only the LSM model was assessed with respect to its absolute 
fit (-at Stage IV) which was obtained. Consequently, the LSM model was 
selected as the preferred model. 

IjfeadLotthe fixajnples..,cpns.ider.ed--i.t may be seen that the decision 
strategy led to the selection of the generating model. 
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